In recent years, multiscale pedestrian detection received extensive attentions in the field of computer vision. In traditional methods, the input image must be resized with different scales to compute the features, which significantly reduces the detection speed. Color Self-Similarity Feature (CSSF) was presented to overcome this problem. An improved CSSF with lower dimension was proposed for the CSSF whose dimension is too high and time-consuming in the training process of the classifiers. Combined with pedestrian structural similarity, a fixed-size window was defined at first, and then the improved CSSF was extracted by sliding the fixed-size window in different color space. Finally, the pedestrian detection classifier was constructed by combining with AdaBoost algorithm. Test shows that compared with the traditional CSSF whose dimension is ten millions, new feature dimension is only a few thousand, and it can be extracted and trained faster, but detection effect decreases slightly; compared with the Histogram of Oriented Gradient (HOG), feature extraction speed improves 5 times, detection effect is essentially the same. The new method has a good application value in real-time pedestrian detection and monitoring systems.